IN THE CLAIMS : 

Please amend Claims 1-4, 7, 9-11, 17-20, and 39 as follows. A marked-up 
copy of the amended claims showing the changes made thereto, is attached. Note that all 
the claims currently pending in this application, including those not presently being 
amended, have been reproduced below for the Examiner's convenience. 

L (Twice Amended) A simulator apparatus with which an operator plays a 
simulation with virtual object(s) in mixed reality space including a virtual space andj:eal 
space, said simulator comprising: 

a viewpoint detection unit adapted to detect the location/posture of a 
viewpoint of the operator; 

an inputting unit adapted to input a real spac^image corresponding to the 
location/posture of a viewpoint of the operator; 

a geometric information acquisit'i6n unit adapted to recognize geometric 
information of real object(s); 

a recognition imit^adapted to recognize a current relative relationship 
between the virtual object(s) and real object(s); 

a rule memory adapted to store rules for controlling the action of the virtual 

object(s); 

a'Computation unit adapted to determine the next action of the virtual 
object(s) in/accordance with the rules stored in said rule memory and in correspondence 
with the'location/posture of the real object(s), and to compute the location/posture of the 
virtual object(s) after the determined action; and 




a presentation unit adapted to genferate atjeast-one image of the virtual 
object(s) on the basis of the location^os^^ virtual object(s) after the determined 

action and the location/posture^of the viewpoint position of the operator, and to represent 
the mixed reality.sp'Sceto the operator by supdnmposing the image(s) of the virtual 
objept('sO on the operator's view of the real space. 



2. (Amended) The apparatus according. to claim 1, wherein said presentation 
unit further comprises: 

an image-capturing unit adapted to capture real space images of said 
operator's view of real space images of said player's view of the real space; 

an image generation unit adipted to generate mixed reality images 
representing the mixed reality space by superimposing or overlaying said image(s) of the 

and 

a video see-through type di|splay that the operator wears wherein said mixed 
reality images are displayed. 

3. (Amended) The apparatus according to claim 1, wherein said presentation 
unit further comprises an optical see-throi gh type display that the operator wears wherein 
said virtual object image(s) are displayed. 



virtual object(s) on said real space images 



4. (Amended) The apparatu 



s according to claim 1, further comprising, 



a status detector that detects a status of the operator; 



wherein said computation unit determines a next action of the virtual object 
in accordance with the rule stored in said nile memory and in correspondence with the 
location/posture of the real object and/or the status of the operator, and computes a 



location/posture of the virtual object after 



le determined action. 



5. (Not Presently Amende] 
the current relative relationship includes 



I) The apparatus according to claim 1, wherein 
layout relationship between the virtual object 



and real object at the current time in the nixed reality space 



6. (Not Presently Amended) The apparatus according to claim 1, wherein 



the current relative relationship includes 



a behavior of the real object with respect to the 



virtual object at the current time in the mixed reality space. 



7. (Amended) The appar; 
includes the operator himself or herself, 
relative relationship between the vj^al o1 



scoi^g-to^ 1, 



tus aecordir^g^ claim 1, wherein the real object 

said recognijidn^unit recognizes a current 
3Jec^afla the operator. 



8. (Not Presently Amended) The apparatus according o claim 1, wherein the 
real object includes a plurality of operators Who operate said simulator apparatus, and the 
plurality of operators share a single mixed reality space. 

9. (Amended) The apparatus Recording to claim 1, wherein the real object 
is an object which is fixed in position, and 
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.A 

iition 



said geometric information acquisition unit comprises: 
a predetermined memory for pre-storing location information and shape 

information of the real object; and 

a reading unit that reads out the location information and shape information 

of the real object from said predetermined memory as needed. 



10. (Amended) The apparatus according to claim.l< wherein the real-object 




is an object which is movable but does not deform, and 



said geometric information acquisition unit comprises 



a predetermined memory^ 



real object; 



a location/pbsture sensor 

/ 



for pre-storing shape information of the 



or detecting a location/posture of the real 



object; and 

^^a^'setting unit that sets a region the real object is expected to occupy 
in/the mixed real space in accordance with the detected location/posture of 



the real object. 



11. (Amended) The apparatus^accqrding-to claim 1, wherein the real object 
is an operator, and 

said geometric informifioii acquisitibn unifcomprises: 
a seiis<5r^fbi^etectm^ 

and 



on/posture of a head of the operator; 
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a setting unit that sets a regio/havirig'a fixed, known shape that 
approximates the operator inttig^ni'^^ reahjy^pace'in accordance with the 
detected location/posji^^f theopp^^ head. 



12. (Not Presently Amended) The apparatus according to claim 1, wherein 
when the simulation is a battle simulation with tfie virtual object, an objective is to 
decrease an.expected score^of the operator. - 



13. (Not Presently Amended) T 
when the simulation is a cooperative simulation 
increase an expected score of the simulation 



le apparatus according to claim 1, wherein 
with the virtual object, an objective is to 



14. (Not Presently Amended) The apparatus according to claim 1, wherein 
one of the rules stored in said rule memory conirols the action of the virtual object on the 
basis of an objective of the simulation and a relative relationship between the virtual object 
and real object. 



15. (Not Presently Amended) The\ apparatus according to claim 1, wherein 
one of the rules stored in said rule memory expresses the action of the virtual object as an 
action pattern with a predetermined aim for achieving an objective of the simulation. 
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16. (Not Presently Amended) The apparatus according to claim 15, wherein 
the pattem has a path disadvantageous to the operator in consideration of a layout 
relationship between the virtual object and real object. 



17. (Amended) The apparatus ac^ordBg to claimj^r-^^^^ when the 
operator is one of the real objects, an outputjfrom smdj/iewpoint^detection unit for 
detecting the location/posture of the vie^ojmit-of thej^^^ as information 



which is to be acquired by said geomei 
location and shape of the onefator. 




fotmation acquisition unit and pertains to a 



18. (Amended) The apparatus according to claim 1, wherein said viewpoint 
detection unit detects a location/posture o 'the head of the operator, and 

said apparatus further comprises a detector that detects a location/posture of 
a hand of the operator; and 

a recognition unit adapted to recognize a relative location of the hand of the 
operator with respect to the head as a con^and on the basis of an output from said 
detector. 




19. (Amended) The apparatus^according to claim 1, wherein said 
^"presentation unit comprises : 

an alignment unit that aligns the location/posture of the real object to the 
location/posture of the virtual object after mpvement; 



a generation unit that generates 
alignment in correspondence with an ocelusioiji 
a head-mounted display device 



an imaee 



image of the virtual object after 
relationship; and 



20. (Tv^ice Amended) An ima^e processing/method for a simulator 



apparatus with which an operator plays a gan: 



space compnsmg; 

a vievv^oint detection step foi/detecting the location/posture of a viewpoint 



e with virtual object(s) in a mixed reality 



\ real space image corresponding to the 



of the operator; 

an inputting step formputting 
location/posture of a viev^oin^f the operator; 

a geometric information acquisition step for acquiring geometric 
information of real object(s); 

a recogiution step for recogniz|ng a current relative relationship between the 
virtual object(s) and real object(s); 

a/^mputation step for determining the next action of the virtual object(s) in 
accordance with the rules stored in a rule membry, which stores rules for controlling the 
action of the virtual object(s), and in correspondence with the location/posture of the real 



object(s); and computing the location/posture o 
action;^d 

a presentation step for generating 
on the basis of the location/posture of the virtua 



me location/posture of the viev^oint position of 



the virtual object(s) after the determined 

at least one image of the virtual object(s) 
object(s) after the determined action and 
he operator, and for representing the 
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^ mixed reality space to the operator b»siiperimposing the image(s) of virtual object(s) on 



the operator's view of the real'^pace. 



21 . (Not Presently Amended) \ ^he method according to claim 20, wherein 
the operator wears a video see-through type di^lay, and said presentation step further 
comprising, 

image-capturing step for captuiing real space images of said operator's view 
of the real space; 

image generation step for generating mixed reality images representing of 
the mixed reality space by superimposing o| overlaying said image(s) of virtual object(s) 
on said real space images and for displaying said mixed reality images on the display. 



22. (Not Presently Amended) The method according claim 20, wherein the 
operator wears an optical see-through type display and said presentation step representing 
the mixed reaUty space to the operator by djsplaying the image(s) of virtual object(s) on the 
display. 



23. (Not Presently Amended)\ The method according to claim 20, further 

comprising, 

status detecting step for detecting status of the operator; 
wherein said computation step determines a next action of the virtual object 
in accordance with the rule stored in said rule nismory and in correspondence with the 
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location/posture of the real object and/or the s ;atus of the operator, and computing a 
location/posture of the virtual object after the determined action. 



24. (Not Presently Amended' 
the recognition step recognizes the current re 
relationship between the virtual object and 
reality space. 



re il 



The method according to claim 20, wherein 
ative relationship including a layout 
object at the current time in the mixed 



25. (Not Presently Amended) The method according to claim 20, wherein 
the recognition step recognizes the current relative relationship including a behavior of the 



real object with respect to the virtual object 



at the current time in the mixed reality space. 



26. (Not Presently Amendec 
the recognition step includes the step of recc^gnizmg 
the virtual object and the operator, and the r^al 
herself. 



The method according to claim 20, wherein 
a current relative relationship between 
object includes the operator himself or 



27. (Not Presently Amended) 
a plurality of operators share a single mixed 
computation step includes a plurality 



The method according to claim 20, wherein 
reality space and the real object(s) used in the 
of operators who operate the apparatus. 



28, (Not Presently Amended) 
the real object is an object which is fixed in p< 



The method according to claim 20, wherein 
sition, and 
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the geometric information acquisition step includes the steps of: 
pre-storing location information and shape information of the real object in 

a predetermined memory; and 

reading out the location informati)3n and shape information of the real object 

from the predetermined memory as needed. 

29. (Unamended) The method according to claim 20, wherein the real 
object is an object which is movable but does not deform, and 

the geometric information acquisition step includes the steps of: 

pre-storing shape information of the real object in a predetermined 
memory; 

detecting a location/posti^re of the real object by a location/posture 
sensor; and 

setting a region the real c 
real space in accordance wit h thle 
object. 



3ject is expected to occupy in the mixed 
detected location/posture of the real 



30. (Not Presently Amended) Ijhe method according to claim 20, wherein 

the real object is an operator, and 

the geometric information acquisition step includes the steps of: 
detecting a location/posture of the head of the operator; and 
setting a region having a fixed, kiown shape that approximates the operator 

in the mixed reality space in accordance with the^etected location/posture of the head. 



3 1 . (Not Presently Amended) The methpd according to claim 20, wherein 
when the simulation is a battle simulation with the virtmal object, an objective used in said 
computation step is to decrease an expected score of th'e operator. 

32. (Not Presently Amended) The method according to claim 20, wherein 
when the simulation is a cooperative simulation witn the virtual object, an objective used 
in said computation step is to increase an expected score of the operator. 

33. (Not Presently Amended) The/method according to claim 20, wherein 
one of the rules controls the action of the virtual object on the basis of an objective of the 
simulation and a relative relationship between the virtual object and real object. 



34. (Not Presently Amended) The method according to claim 20, wherein 
one of the rules stored in the rule memory expresses the action of the virtual object as an 
action pattern with a predetermined aim for achieving the objective of the simulation. 



35. (Not Presently Amended) 



The method according to claim 34, wherein 



said computation step determines an action using a pattern having a path disadvantageous 
to the operator in consideration of a layout relationship between the virtual object and real 
object. 



36. (Not Presently Amended) 
when the operator is one of real objects, said 



The method according to claim 30, wherein 
geometric information acquisition step uses 



V 
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an output from said viewpoint detection step 
viewpoint of the operator that pertains to a locatioi 



of detecting the location/posture of the 
n and shape of the operator. 



37. (Not Presently Amended) The method according to claim 20, wherein 
the viewpoint detection step includes the step off detecting a location/posture of the head of 
the operator , and 

said method further comprises thje detection step of detecting a 
location/posture of a hand of the operator; and 

the step of recognizing a relative location of the hand of the operator with 
respect to the head as a command on the basis of an output in the detection step. 



38. (Unamended) The method Recording to claim 20, wherein the 
presentation step includes the steps of: 

aligning the location/posture of the real object to the location/posture of the 
virtual object after movement; and 

generating an image of the virtua 
with an occlusion relationship. 



object after alignment in correspondence 



39. (Twice Amended) A storage 



. storage mea 
1 



edium which stores a program of an 



image processing method for a simulator apparatus with which an operator plays a 
(u^ simulation with virtual object('s) in a mixed reality space including a virtual space and real 

\ 

space comprising: y 
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